Switch from `rust-toolchain` to `rust-toolchain.toml` so we can specify components #520

kkysen · 2022-07-13T16:37:58Z

This switches all rust-toolchains to rust-toolchain.tomls so we can specify components (and targets, etc. where needed).

The benefit of this is that cargo will automatically download the needed components and targets instead of erroring, which is very useful when upgrading our nightly. CI breaks when this happens, and we can't simultaneously support multiple nightly versions (across branches and PRs) as we migrate. It is still useful for the Docker containers to pre-install them, as that saves on re-downloading them every time, but it should prevent hard errors. Furthermore, it is useful when developing locally as well.

This should (hopefully) make #513 pass CI.

Previously, we would emit a rust-toolchain file in the transpiled crate if nightly features we needed. That was simple with a rust-toolchain file, but with rust-toolchain.toml, it's more complex, as we don't want to override fields in an existing rust-toolchain.toml that we don't have to. As this is more complex and isn't currently needed for our existing tests and CI, I'll fix it in a future PR that merges/updates (i.e. override the channel, append to components and targets, etc.) the rust-toolchain.toml instead of wholly replacing it.

c2rust/c2rust-transpile/src/build_files/mod.rs

Lines 259 to 265 in a91e816

    
           /// If we translate variadic functions, the output will only compile 
        
           /// on a nightly toolchain until the `c_variadics` feature is stable. 
        
           fn emit_rust_toolchain(tcfg: &TranspilerConfig, build_dir: &Path) { 
        
               let output_path = build_dir.join("rust-toolchain.toml"); 
        
               let output = include_str!("../../rust-toolchain.toml").to_string(); 
        
               maybe_write_to_file(&output_path, output, tcfg.overwrite_existing); 
        
           }

docker/Dockerfile

kkysen · 2022-07-13T17:14:44Z

This is failing CI due to the added toml dependency in requirements.txt. The tests pass locally after doing pip install -r requirements.txt. Once the Docker containers are updated, this should work again.

scripts/docker_build.sh

scripts/provision_mac.sh

scripts/common.py

fw-immunant

Issues commented inline.

kkysen · 2022-07-13T17:33:47Z

I thought I had fixed the parsing issues somewhere, but I guess I left that in a stash somewhere along with the toml merge changes.

scripts/common.py

kkysen · 2022-07-13T18:37:03Z

@thedataking, this does need a Docker update for the toml Python dependency actually. Then this and #513 will work.

…lchain.toml` format.

…l using the `toml` library to parse.

I still left them in `provision_rust.sh` as well, even though they're not strictly needed, because it will help to have them cached in the Docker image. Ideally, I think we'd put c2rust itself (or at least parts of it) into the Docker image as well for caching purposes.

kkysen · 2022-08-09T05:23:46Z

@thedataking, CI is still failing on import toml meaning pip install -r requirements.txt still run and get into the pushed Docker containers.

scripts/common.py

…st-toolchain.toml`. I believe they should be in sync because this one is produced by copying the root one.

…chain.toml` file now.

…`query_toml.py`, which is also a CLI and is used in `docker_build.sh` and `provision_mac.sh` to query the channel now.

… `toml-edit` (as `cargo` does).

…s merged after I wrote #520.

kkysen requested a review from fw-immunant July 13, 2022 16:38